Skip to content

feat: ad hoc filter finishing touches - #3658

Open
knudtty wants to merge 5 commits into
mainfrom
aaron/adhoc-follow-up
Open

knudtty wants to merge 5 commits into
mainfrom
aaron/adhoc-follow-up

Conversation

@knudtty

@knudtty knudtty commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Finishing touches for ad hoc dashboard filters, which are still behind NEXT_PUBLIC_ENABLE_ADHOC_FILTERS.

  • The name is an identifier, not a variable. You can't reference an ad hoc filter as $name, but its name still has to be unique because it keys the filter's conditions in the filters= URL param. The edit form now says so. The filter-bar tooltip and the filters list no longer present it as a variable. getDashboardVariableFilters skips ad hoc filters, so checkAlerts, MCP validation and the alert chart stop declaring them. isVariableEnabled: true stays, because name-uniqueness validation reads it.
  • Broadcast is always on. With no variable, broadcasting is the only way the conditions apply. The schema now requires isBroadcastEnabled: z.literal(true), the same way STATIC_LIST and PROMETHEUS_LABEL use z.literal(false). The edit form drops the broadcast checkbox and always shows "Applies to sources".
  • Overlap warning. The edit form shows a warning when another ad hoc filter of the same source type already applies to any of the same sources. A click-to-filter on such a tile updates every one of those filters, which is confusing. It is a warning only; saving still works.
  • Collapsed chips. The filter input shows the first 3 conditions plus a "+N more" toggle that expands in place. Series with many labels, which is common in PromQL, made the input grow very tall.
  • IN (SQL) and =| (PromQL) operators. Each condition still holds one key and one value. When rendered, conditions sharing a key are OR'd: SQL renders toString(key) IN ('a', 'b') and PromQL renders key=~"a|b", with regex characters escaped. These are two separate stored values, each valid only for its source type. A condition the filter's source type can't express is skipped and its chip is marked invalid.

Compatibility: an ad hoc filter already stored with isBroadcastEnabled: false, or without the field, will no longer parse. The feature is behind a flag, so there is no migration.

Screenshots or video

I couldn't capture screenshots from my environment. @aaronknudtson, please add before/after shots of the ad hoc filter edit modal and of a filter input holding more than 3 conditions.

Before After

How to reproduce

Starting point: yarn dev. packages/app/.env.development sets NEXT_PUBLIC_ENABLE_ADHOC_FILTERS=true.

On main:

  1. Open a dashboard, click "Filters", then "Add filter", and choose the type "Ad hoc keys and values".
  2. The form has a "Broadcast filter conditions" checkbox, and the "Variable name" tooltip says it's how the variable is referenced.
  3. Save a filter over the Logs source, then add a second one over Logs. No warning appears.
  4. Add 6 conditions to one filter. The input wraps onto several rows.
  5. Open the operator select in the condition editor. There is no in option.

On this branch:

  1. In the same form there is no broadcast checkbox, and "Applies to sources" is always visible. Under "Variable name" it reads "Unique name for this filter. Ad hoc filters can't be referenced as variables in tile queries."
  2. Adding a second ad hoc filter over Logs shows "Another ad hoc filter (…) already applies to these sources."
  3. With 6 conditions, the input shows 3 chips and "+3 more". Clicking it shows every chip and "Show less".
  4. Add ServiceName in api and ServiceName in web. Tiles on Logs filter with toString(ServiceName) IN ('api', 'web'). On a Prometheus ad hoc filter, =| conditions on the same label render as label=~"(a|b)".
  5. Hovering the filter label shows "Filters N sources", without "available as variable".

How to test on Vercel preview

N/A. The preview builds without NEXT_PUBLIC_ENABLE_ADHOC_FILTERS, so ad hoc filters aren't offered there.

Preview routes: N/A

Steps:

  1. N/A

References

  • Linear Issue: Closes HDX-5761

- Add IN (SQL) and =| (PromQL) operators; conditions on the same key are OR'd
  into one IN (...) or =~"a|b" condition.
- Ad hoc filters always broadcast (isBroadcastEnabled: z.literal(true)), and
  their variable name is a unique identifier only, no longer declared as a
  $variable.
- Warn when another ad hoc filter already applies to the same sources.
- Collapse the filter input to three conditions behind a "+N more" toggle.
@knudtty
knudtty requested a review from a team as a code owner November 13, 2026 21:17
@changeset-bot

changeset-bot Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 0b8dfa4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Patch
@hyperdx/app Patch
@hyperdx/api Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 16, 2026 3:00pm UTC
hyperdx-storybook Ready Ready Preview Nov 16, 2026 3:00pm UTC

Request Review

@github-actions github-actions Bot added the review/tier-3 Standard — full human review required label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Diff size: 399 production lines changed (Tier 2 max: < 250)
  • Cross-layer change: touches frontend (packages/app) + shared utils (packages/common-utils)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 9
  • Production lines changed: 399 (+ 369 in test files, excluded from tier calculation)
  • Branch: aaron/adhoc-follow-up
  • Author: knudtty

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 108 in total (1.18%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

4 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 4 minor

3 posted as inline comment(s) on the changed lines. 1 listed below.

Findings outside the changed lines

1 minor
  • 🔵 packages/app/src/hooks/useDashboardFilters.tsx:241 — Redundant ad hoc exclusion now that getDashboardVariableFilters skips ADHOC → getDashboardVariableFilters now applies isFilterReferenceableVariable, which already drops ADHOC filters, so passing nonAdhocFilters here filters them twice. The comment at line 212 ("Ad hoc filters aren't exposed as variables yet") is also stale. Pass filters, or drop the comment, so only one place excludes ad hoc filters.

Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

Scope: feat: ad hoc filter finishing touches (PR #3658) against base c220e164 — 9 production files, ~399 production lines, frontend (packages/app) + shared utils (packages/common-utils). Feature is behind NEXT_PUBLIC_ENABLE_ADHOC_FILTERS.

Intent: Make the ad hoc filter name a unique identifier rather than a referenceable variable, force broadcast always-on, add an overlap warning, collapse condition chips, and add IN (SQL) / =| (PromQL) operators that OR together conditions sharing a key.

The core logic is sound: escaping for both new operators is correct (SQL IN reuses escapeSqlString; PromQL =| composes escapeRegexValue then escapePromqlStringValue in the right order, verified against the a.b\c test case), the collapsed-chip slice keeps a 0-based prefix so remove(index)/replace(index) stay aligned, unsupported operators are pre-filtered before the defensive throw, and the isFilterReferenceableVariable split preserves name-uniqueness via isFilterVariableEnabled. Test coverage for the new behavior is thorough.

🟡 P2 — recommended

  • packages/common-utils/src/types.ts:1344 — SQL_ADHOC_FILTER_OPERATORS / PROMQL_ADHOC_FILTER_OPERATORS changed from being derived off AdhocFilterOperatorSchema.options to hand-maintained literal subsets, so adding a future operator to the enum silently classifies it as unsupported for both source types with no compile-time guard.
    • Fix: Derive each list from AdhocFilterOperatorSchema.options by exclusion, or back both with an exhaustive Record<AdhocFilterOperator, {sql: boolean; promql: boolean}> so every new enum member must be categorized at compile time.
    • kieran-typescript, maintainability
  • packages/common-utils/src/types.ts:2452 — Tightening isBroadcastEnabled to z.literal(true) makes any ad hoc filter previously persisted with isBroadcastEnabled: false/absent fail the strict AdhocDashboardFilterSchema; because dashboards parse z.array(DashboardFilterSchema) (types.ts:2527, 2571), one such filter fails the whole dashboard's filter parse on load for anyone who saved one under the preview flag.
    • Fix: Coerce isBroadcastEnabled to true for ADHOC filters on read (a z.preprocess or tolerant default) instead of hard-rejecting, so pre-flag dashboards still load.
    • correctness, kieran-typescript
🔵 P3 nitpicks (5)
  • packages/common-utils/src/filters.ts:1062 — The "one-of" concept is encoded as two operator values (IN for SQL, =| for PromQL) and that source-type correspondence is repeated across ONE_OF in filterOperators.ts, the literal args to groupAdhocConditionsByKey, the render switches, and the operator lists, with no single source of truth.
    • Fix: Introduce one ONE_OF_OPERATOR_BY_SOURCE = { sql: 'IN', promql: '=|' } map and derive the grouping operator and shared option from it.
  • packages/common-utils/src/filters.ts:873 — isFilterReferenceableVariable and isFilterVariableEnabled are near-identical names whose real distinction (tile-referenceability vs name-uniqueness/storage keying) isn't evident, and untouched call sites still use the latter.
    • Fix: Rename the uniqueness predicate to name its axis explicitly (e.g. hasUniqueFilterName) so each call site self-documents.
  • packages/app/src/components/DashboardFiltersModal/VariableNameInput.tsx:15 — VariableNameInput now takes three cooperating props (label, description, noun) whose only coherent combination is the ad hoc "not a variable" copy, which callers must keep mutually consistent.
    • Fix: Collapse them into a single referenceable?: boolean mode flag that selects the label/description/noun set internally.
  • packages/app/src/components/AdhocDashboardFilter/__tests__/AdhocDashboardFilter.test.tsx:85 — No test removes or edits a pill while the chip list is collapsed (>3 conditions), so the prefix-slice index invariant is unguarded against a future windowed-slice refactor.
    • Fix: Add a collapsed-state case that removes/edits a visible pill and asserts onChange receives the correct original-array index.
  • packages/app/src/components/AdhocDashboardFilter/__tests__/AdhocDashboardFilter.test.tsx:109 — The pill invalidReason now interpolates languageName (SQL vs PromQL), but the invalid-pill test only asserts data-invalid, not the reason text.
    • Fix: Assert the invalid-reason text names the correct source language.

Reviewers (7): correctness, security, kieran-typescript, testing, maintainability, project-standards, previous-comments.

Testing gaps:

  • No test exercising remove/replace of a visible pill while chips are collapsed (index correctness relies on the prefix-slice invariant).
  • No test asserting the SQL/PromQL substitution in the invalid-pill reason text.
  • No test switching an ad hoc filter's sourceType so IN/=| conditions become unsupported and are silently dropped while shown as invalid pills.
  • No test characterizing load behavior for a legacy ADHOC filter with isBroadcastEnabled !== true.

Coverage note: The adversarial, api-contract, and agent-native reviewers were dispatched but had not returned when synthesis was required; the schema-parse blast radius they would have examined was independently verified by the orchestrator. Security review found no new injection exposure (keys are interpolated verbatim in both new and pre-existing branches — a pre-existing, not newly introduced, consideration).

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 468 passed • 1 skipped • 1156s

Status Count
✅ Passed 468
❌ Failed 0
⚠️ Flaky 0
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

Comment thread packages/app/src/DashboardFilters.tsx Resolved
One predicate decides whether a filter's name is a referenceable variable,
used by getDashboardVariableFilters, getFilterEffect, and the filters list.
Updates the ad hoc filter e2e specs for the removed broadcast checkbox and
the list no longer showing an ad hoc filter's name as a variable.
Comment thread packages/common-utils/src/filters.ts Resolved
Comment thread packages/common-utils/src/types.ts Resolved

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 0b8dfa4c Deployed Nov 16, 2026 by vercel[bot]
Preview – hyperdx-storybook — 0b8dfa4c Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant